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1.  EXECUTIVE  SUMMARY. 


1.1  OBJECTIVE. 

The  Federal  Aviation  Administration  (FAA)  uses  mathematical  models  to 
measure  and  predict  the  reliability  of  hardware.  FAA  engineering 
specifications  for  systems  under  development  contain  reliability 
requirements,  usually  in  terms  of  mean-time-Lotwcen-failures  (MTBF) , 
for  the  hardware  portions  of  these  systems.  However,  recently  pro¬ 
cured  systems  contain  computers  with  copious  amounts  of  software. 

It  has  been  the  experience  of  the  data  processing  community  that 
failures  of  such  systems  are  not  confined  to  hardware.  Software  which 
has  been  debugged  and  in  use  for  many  years  has  been  known  to  cause 
system  failure.  Consequently  the  FAA  initiated  this  pilot  study  to 
determine  the  magnitude  of  the  software  reliability  problem  in  one 
system  currently  in  development.  Study  objectives  were  the  following: 

-  Develop  a  software  reliability  model  for  the  Discrete  Address 
Beacon  System  (DABS)  and  make  a  software  reliability  prediction. 

-  Review  and  critique  the  available  hardware  reliability  model 
and  the  hardware  reliability  prediction  for  DABS. 

-  Integrate/evolve  the  software  and  hardware  reliability  models 
into  a  DABS  system  model  and  make  a  system  reliability  prediction. 

-  Compare  the  predicted  systems  reliability  value  versus  the 
specified  value.  Make  applicable  recommendations  for  reliability 
improvement  of  the  system. 

-  Recommend  a  software  reliability  failure  reporting  system  for 
the  DABS. 


1.2  BACKGROUND. 

The  objectives  cited  above  were  accomplished  by  grouping  related 
objectives  and  tasks  accordinj  to  importance  as  defined  by  Mr.  G. 
Apostolakis,  head  of  the  Reliability  Engineering  Section  at  the  FAA 
Technical  Center.  As  stated  by  Mr.  Apostolakis,  the  primary  concern 
of  the  FAA  is  the  study  of  DABS  software  reliability — how  it  could  be 
measured,  modeled,  predicted  and  how  it  could  be  incorporated  with  the 
hardware  into  an  integrated  software/hardware  reliability  model  for 
DABS.  The  results  of  this  study  are  contained  in  the  body  of  this 
report  ,  Sections  2  through  5.  Of  secondary  concern  are  1)  the  review 
and  critique  of  the  DABS  hardware  reliability  model  and  prediction, 
and  2)  a  recommended  software  reliability  failure  reporting  system  for 


the  DABS.  A  brief  critique  of  the  DABS  hardware  reliability  model  is 
contained  in  Appendix  A  to  this  report.  Because  the  FAA  already  has 
a  failure  reporting  system  for  DABS  software,  a  review  of  the  proce¬ 
dures  and  forms  was  made.  Recommendations  for  improvement  are  con¬ 
tained  in  Appendix  B  to  this  report. 

The  DABS  software  reliability  was  modeled  using  test  time  and  failure 
data  obtained  from  the  testing  of  the  sensors  at  three  test  sites--FAA 
Technical  Center,  Elwood  and  Clementon,  N.,1.  Based  on  tests  conducted 
between  February  1979  and  June  1980,  reliabilitv  measurements  were  made 
for  nine  software  modules  which  comprise  the  DABS  mission  software. 
Maintenance  and  off-line  software  were  not  modeled.  Also  not  modeled 
was  the  Automatic  Traffic  Advisory  and  Resolution  Service  (ATARS) 
module  because  of  its  interim  status. 

Reliability  prediction  models  for  software  modules  were  derived  and 
then  verified  by  matching  predictions  of  error  rate  with  software 
test  data  collected  during  July,  August  and  September  of  1980.  Meas¬ 
urements  of  software  reliability  obtained  from  the  models  were  com¬ 
bined  with  hardware  reliability  predictions  (prepared  by  FAA)  to 
obtain  an  integrated  DABS  reliability  prediction  model. 

":.f>  :  *  unately ,  there  is  no  concensus  in  the  literature  pertaining  to 
the  definitions  of  commonly  used  terms  such  as  bugs,  errors,  faults 
and  failures.  A  few  definitions  are  presented  here  to  provide  the 
leader  some  insight  into  those  terms  and  concepts  of  software  reli- 

;  i  ]  i  t  V  . 


-  Software  bugs,  errors  and  faults  will  be  considered  to  be 
synonymous.  They  denote  latent  defects  present  in  software  due?  to 
coding  errors,  misunderstanding  of  the  required  logic  on  the  part  of 
the  i  rogrammer,  incorrect  algorithms  or  other  programming  errors. 

-  A  software  failure  occurs  when  certain  combinations  of  inj ut 

; at  .t  meters ,  input  commands,  input  options  or  input  data  exercise  the 
defective  jart  of  the  program.  Under  a  large  variety  of  ci  i  cw:;l  t  aruvs, 
one  may  consider  these  inputs  to  be  random  sets  from  all  possible 
in juts.  These  random  sets  of  inputs,  in  turn,  cause  random  failures 
iri  the  corresponding  outputs.  The  random  output  failures  may  be  ana - 
1'.  .'.el  statistically  and  thus  constitute  the  basis  for  the  concept  of 
it  liability  as  applied  to  software  failures. 

-  Software  reliability  is  the  probabil i ty  that  a  given  software 
i  loiji.tin  will  operate  without  failure  for  a  specified  time  in  a 

: i  c  i  f  i  ed  envi  ronment . 
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1.3  SIGNIFICANT  FINDINGS. 


1.  The  DABS  has  a  measured  overall  MTBF  of  252  hours;  575  hours  MTBF 
for  the  hardware  and  448  hours  for  the  software.  Only  critical  fail¬ 
ures,  those  which  dramatically  reduce  system  capability,  were  counted 
in  computing  the  MTBFs  of  hardware  and  software. 

This  achieved  MTBF  is  far  short  of  the  1000-hour  MTBF  specified  in 
the  engineering  requirements .  It  is  recognized  that  the  required 
MTBF  of  1000  hours  was  intended  for  application  only  to  hardware, 
but  even  if  the  software  is  ignored  the  system  does  not  now  meet  its 
reliability  requirement.  If  all  chargeable  software  failures  were 
included  in  the  calculation,  software  MTBF  would  decrease  to  81  hours 
and  the  system  MTBF  would  decrease  to  71  hours.  These  measurements 
are  based  on  a  total  of  5386  software  test  hours  during  which  354 
errors  were  observed  and  evaluated. 

It  should  be  recognized  that  DABS  is  undergoing  development  testing 
and  that  its  reliability  is  expected  to  increase  as  improvements  are 
incorporated.  Also,  the  transition  from  a  test  or  debugging  scenario 
to  an  operational  scenario  should  noticeably  improve  the  measured  MTBF 
of  the  software.  Much  of  the  software  testing  at  the  Technical  Center 
was  geared  toward  pushing  the  system  to  its  specified  operational 
limits  (o.q., capacity  testing,  multiple  correlations,  crossing  tracks). 
The  system  was  tested  using  a  multitude  of  input  environments  and 
many  of  the  reported  errors  were  discovered  as  a  result  of  testing 
using  input  environments  which  would  not  ordinarily  be  encountered 
in  an  operational  scenario. 

2.  A  critical  software  failure  will  frequently  have  a  far  greater 
effect  on  system  operation  than  a  computer  hardware  failure  because 
critical  software  failures  cause  a  significant  or  complete  loss  of 
system  capability;  that  is,  they  defeat  the  hardware  redundancy  built 
into  the  system.  In  the  event  of  computer  failure  the  system  can 
recover  by  using  a  spare  computer ;  however ,  critical  software  ‘  ai lures 
result  in  either  complete  system  failure  or  reduced  per formanco  which 
does  not  meet  specification.  From  a  reliability  point  of  view, 
partial  system  operation  is  considered  to  be  a  failed  condition 
because  no  reliability  requirements  are  specified  for  alternate 
(degraded)  modes  of  operation. 

It  is  recommended  that  the  FAA  investigate  the  design  of  fault- 
tolerant  software  for  DABS.  The  software  could  be  designed  to  sense 
critical  software  failures  (watchdog  logic  or  audits)  which  would 
recover  the  system  in  much  the  same  fashion  as  a  computer  failure 
by  causing  an  automatic  re-initialization  of  the  system. 
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3.  The  Duane  reliability  growth  model  which  has  been  used  extensively 
to  model  the  growth  of  hardware  reliability  and  more  recently  to  model 
the  growth  of  software  reliability  as  well,  fits  the  known  history  of 
DABS  software.  Of  the  nine  software  modules  in  DABS,  the  Duane  model 
accurately  predicts  reliability  and  rate  of  reliability  growth  of 
five  modules.  The  modules  and  their  rate  of  reliability  growth  models 
are : 


-.503 

Communication:  X£  =  .174  T 

Measured  MTBF  at  end  of  study:  976  hours 

-.419 

Performance  Monitoring:  X^  =  .1403  T 
Measured  MTBF  at  end  of  study:  494  hours 

-.645 

Message  Routing  &  Data  Link:  X^  =  .3467  T 

Measured  MTBF  at  end  of  study:  2400  hours 

System  Software:  X£  =  5.689  T  * 

Measured  MTBF  at  end  of  study:  2588  hours 

c  -ii  i  me  t  'r--  3071 

Surveillance:  Xr  =  .1067  f 

Measured  MTBF  at  end  of  study:  207  hours 

where  X£  =  cumulative  error  rate  (number  of  chargeable  errors/total 
test  hours)  and  T  =  cumulative  test  hours.  Of  the  remaining  modules, 
the  data  were  either  too  sparse  to  evaluate  the  parameters  of  the 
model  or  too  erratic  to  determine  whether  module  reliability  is 

Lm;  roving . 

The*  parameters  appearing  as  exponents  of  T  indicate  the  rate  of  MTBF 
growth  (MTBF  =  1/error  rate),  which  is  usually  a  measure  of  manage¬ 
ment  pressure  to  find  and  correct  errors.  The  rates  shown  are  all 
within  the  range  typically  encountered  but  they  vary  more  than  usual. 
This  suggests  that  testing,  debugging  and  integration  efforts  have 
not  been  applied  uniformly  in  the  DABS  program.  Some  modules  have 
received  much  more  attention  than  others. 

4.  Based  on  hardware  reliability  measurements  reported  in  Report  No. 
FAA-RD-80-36,  "Discrete  Address  Beacon  System  (DABS)  Baseline  Test  and 
Lval uat ion, "  April  1980,  DABS  hardware  MTBF  growth  rate,  albeit  using 
a  small  sample,  was  calculated  to  be  a  =  .36.  Projections  of  hard¬ 
ware  MiBF  improvement  using  a  =  . 36  and  software  MTBF  improvement  using 
t  =  .52  show  that  the  DABS  software/hardware  system  will  achieve  its 
1000-hour  MTBF  requirement  after  49  additional  months  of  testing. 

At  that  time  hardware  MTBF  will  be  approximately  1650  hours  and  soft¬ 
ware  MIBF  will  be  approximately  2500  hours  if  the  growth  rate 
■'  : .  t  i  rules  . 
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The  models  predict  that  if  no  changes  are  made  in  the  present  reli¬ 
ability  improvement  efforts,  software  errors  will  still  constitute  10 
percent  of  total  system  errors  (based  on  1000-hour  hardware  MTBF)  after 
50  x  106  software  test  hours  (test  time  needed  to  achieve  software 
MTBF  =  10,000  hours). 

The  following  actions  are  recommended  to  speed  up  the  reliability 
improvement  of  the  DABS  system: 

-  Increase  the  intensity  of  the  software  test  program  to  con¬ 
duct  well  planned  non-random  testing  such  as  the  identification  and 
evaluation  of  degraded  as  well  as  complex  inputs  to  software  modules. 

-  Automatically  identify/isolate  access  of  the  code  with  low 
input/output  traffic;  check  all  jump  statements. 

-  Conduct  failure  modes,  effects  and  criticality  analyses  of 
hardware  elements  which  contribute  most  to  unreliability--antenna , 
transmitter,  receiver  and  processor.  These  elements  which  have  no 
redundancy  in  the  single  channel  sensor  could  be  improved  through  the 
idenfication  of  failure  modes  and  their  elimination  through  correc¬ 
tive  action. 

-  The  reliability  engineering  department  of  the  FAA  Technical 
Center  should  continue  to  monitor  the  progress  of  the  software  test, 
participate  in  configuration  management  and  include  estimates  of 
software  reliability  in  DABS  predictions. 

5.  Analysis  of  the  test  data  for  the  purpose  of  measuring  reliability 
and  its  growth  showed  that  the  error  rates  of  most  software  modules 
changed  appreciably  during  the  test.  Several  causes  are  postulated: 

a)  As  noted  in  Figure  4,  the  nearly  constant  error  rate  of  the 
communication  module  for  900  test  hours  is  followed  by  a  rapidly 
declining  error  rate.  This  pattern  is  characteristic  of  an  early 
test  period  in  which  the  software  package  was  not  tested  with  the 
intensity  needed  to  identify  and  correct  errors.  Subsequent  testing 
then  resulted  in  the  identification  and  elimination  of  more  errors  at 
a  significantly  higher  rate. 

b)  Software  test  personnel  are  sometimes  reluctant  to  document 
errors  as  they  are  observed  because  they  believe  that  continuation 
of  the  test  and  analyses  of  results  are  more  important  tasks.  Con¬ 
sequently,  failures  may  be  documented  en  masse  several  weeks  or 
months  after  they  occur,  usually  at  the  completion  of  the  test. 

Such  perturbations  to  the  model  may  require  several  additional  data 
points  to  effect  smoothing. 
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c)  Neither  the  Duane  nor  any  other  model  which  assumes  a  con¬ 
tinuously  decreasing  error  rate  with  time  can  predict  significantly 
large  perturbations  due  to  mass  introductions  of  software  modifica¬ 
tions  at  "release"  points.  These  can  either  increase  or  decrease  an 
error  rate.  Abrupt  termination  of  a  debugging  process  will  also  sig¬ 
nificantly  reduce  the  observed  error  rate. 

6.  The  FAA  should  endeavor  to  include  software  as  well  as  hardware 
elements  in  future  reliability  models  for  DABS  and  other  computer 
aided  systems.  Reliability  requirements  of  future  systems,  which 
are  often  set  by  systems  requirements  analyses,  should  also  include 
the  reliability  of  the  software.  Measured  reliability  of  the  system 
will  then  be  realistic  since  it  will  apply  to  software  and  hardware. 


2. 


DESCRIPTION  OF  DABS  COMPUTER  SOFTWARE,  TESTING  AND  DATA  BASE. 


2.1  DESCRIPTION  OF  DABS  COMPUTER  SOFTWARE  AND  ITS  TESTING. 


As  described  by  Dr.  C.  M.  Applewhite  in  "Disbributed  Computer  Architec¬ 
ture  For  The  Discrete  Address  Beacon  System,"  the  purpose  of  DABS  is  to 
provide  highly  reliable  tracking  and  collision  avoidance  support  for 
DABS-equipped  aircraft.  Control  of  DABS  is  provided  by  software 
operating  in  a  ground  based  distributed  computer  network  interfaced 
to  a  beacon  radar.  Each  DABS  aircraft  is  assigned  a  unique  identifica¬ 
tion  (discrete  address)  associated  with  its  DABS  transponder.  Recog¬ 
nition  of  a  beacon  interrogation  is  keyed  to  the  discrete  address  of 
each  particular  aircraft  such  that  a  unique  data  link  with  miminum 
interference  can  be  established  between  the  computer  network  and  each 
aircraft.  The  software  subsystem  maintains  a  track  update  on  each 
aircraft,  predicts  potential  conflict  situations  and  controls  the 
scheduling  of  the  beacon  radar.  Data  to  support  aircraft  tracking  is 
gathered  via  uplink  interrogations  and  downlink  responses  of  aircraft 
positional  data.  Traffic  data  and  maneuver  advisories  are  provided 
to  the  pilots  via  the  uplink  in  the  event  the  computer  subsystem 
predicts  a  potential  conflict  situation.  Telephone  line  data  links 
between  sensors  facilitate  coordination  among  adjacent  sensors  with 
overlapping  airspace  responsibilities. 

DABS  surveillance  capability  is  designed  to  be  completely  compatible 
with  the  present  Air  Traffic  Control  Radar  Beacon  System  (ATCRBS)  and 
thus  can  be  introduced  gradually  and  economically  without  major  oper¬ 
ational  or  procedural  change.  Since  DABS  uses  monopulse  direction 
finding,  the  system  also  provides  improved  surveillance  coverage  for 
ATCRBS  equipped  aircraft  at  a  reduced  interrogation  rate. 

In  addition  to  the  requirements  given  above,  the  software  system  is 
required  to  respond  to  computer  hardware  failures  by  reconfiguring 
the  system  and  maintaining  system  integrity,  to  monitor  system  status 
indicators,  to  send  status  messages  to  ATC  maintenance  facilities  and 
to  collect  performance  data  for  the  sensor.  A  functional  block  dia¬ 
gram  which  highlights  some  of  the  DABS  features  is  shown  in  Figure  1. 
The  architecturally  distr ibuted ,  molecular  software  is  shown  in  Fig¬ 
ure  2 . 

No  special  tests  were  run  expressly  to  provide  data,  solely  for  reli¬ 
ability  analysis.  Consequently,  the  running  time  and  errors  generated 
during  debugging,  checkout  and  operation  of  the  DABS  sensors  at  FAA 
Technical  Center,  Elwood  and  Clementon,  N.J.,  were  used  to  formulate 
the  software  reliability  model  and  measure  achieved  reliability  and 
growth  rate.  The  DABS  software  was  tested  formally  and  informally. 

At  the  FAA  Technical  Center,  initial  testing  was  conducted  by  running 


Figure  2.  DABS  Software  Independent  Task/Processor  Organizati 


the  system.  Maximum  specified  values  of  targets  and  fruit  rates 
(replies  to  interrogations  from  other  sensors)  were  simulated  to  test 
DABS  software  and  hardware.  The  test  program  uncovered  coding  errors, 
timing  and  interrupt  faults. 

2.2  DABS  SOFTWARE  DATA  BASE . 

The  software  test  time  base  is  shown  in  Figure  3.  The  figure  contains 
monthly  estimates  of  software  test  time  for  three  facilities  where 
testing  occur red--FAA  Technical  Center,  Elwood  and  Clementon,  NJ. 
Beginning  in  October  1979  the  test  time  is  the  scheduled  software  test 
time  for  each  test  site.  Prior  to  October,  software  test  times  were 
based  on  estimates  obtained  from  Messrs.  M.  Holtz,  DABS  Program  Manager, 
and  J.  Simpfonderfer,  T.  I.  Technical  Representative.  The  figure  also 
shows  the  time  phasing  of  the  testing  of  the  various  DABS  software 
modules.  It  shows,  for  example,  that  channel  management,  surveillance 
and  data  extraction  were  considered  to  be  undergoing  test  from  the 
beginning  of  the  test,  whereas  network  management  was  not  tested  until 
all  three  sensors  were  on-line  in  October  1979. 

An  additional  three  months  of  test  data  (1790  hours)  accrued  during 
duly,  August, and  September  1980.  Ill  is  time  period  constituted  a 
small  controlled  sample  from  DABS  testing  to  be  used  to  verify  relia¬ 
bility  predictions  made  using  the  data  base  described  above.  This 
procedure  is  described  in  Section  3  of  this  report. 

Software  errors  discovered  during  the  test  programs  were  reported  on 
trouble  reports.  A  brief  description  of  the  reporting  system, 
including  a  sample  form,  is  given  in  Appendix  B  to  this  report. 

'Hie  DABS  software  error  data  base  consists  of  354  trouble  reports  (TR) 
which  document  program  stops,  errors,  enhancements  and  change  pro¬ 
posals.  Not  all  supply  adequate  information  for  reliability  analysis. 
Software  design  engineers  at  Texas  Instruments  reviewed  each  TR  and 
its  associated  follow-up  documentation  and  classified  the  TRs  with 
respect  to  severity.  Chargeable  errors  which  were  used  to  measure 
software  reliability  were  classified  as  critical  (1)  or  non-critical 
(2).  Those  not  chargeable  were  classified  other  (3)  or  no  count  (4). 
The  following  definitions  were  applied. 

Chargeable  Errors: 

1.  Critical  -  An  error  in  the  software  which  causes  a  sig¬ 
nificant  or  total  loss  of  operational  system  capability. 

2.  Non-Critical  (Major)  -  An  error  in  the  software  which 
causes  an  erroneous  response  in  the  operational  system.  An  error  in 
this  classification  may  not  be  recognized  as  such  by  a  trained 
observer  due  to  the  self-repair  inherent  in  the  system. 
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Figure  3.  DABS  Software  Test  Baseline 


Non-Chargeab le  Errors: 


3.  Other  (Minor)  -  An  error  which  has  no  measurable  effect 
on  the  operational  system  or  is  of  unknown  cause  at  this  time  (hard¬ 
ware/software/cockpit).  Errors  of  unknown  causes  would  be  charged 
against  the  DABS  system  rather  than  the  software. 

4.  No  Count  -  A  trouble  report  which  was  erroneously  attrib¬ 
uted  to  software  errors.  In  addition,  change  proposals,  enhancements, 
duplicate  trouble  reports  and  "cockpit  errors"  are  included. 

A  summary  of  chargeable  errors  is  presented  in  the  matrix  in  Table  1. 
Only  software  modules  identified  in  the  table  were  included  in  the 
reliability  analysis.  Other  software  modules  which  are  of f-1 inc analy¬ 
sis  tools  or  are  used  during  maintenance  or  pre-initialization  were 
not  modeled  because  they  are  not  part  of  the  mission  software.  The 
ATARS  module  which  will  eventually  constitute  a  large  portion  of  the 
DABS  software  subsystem  cannot  be  analyzed  now  because  it  is  being  re¬ 
written  and  is  not  scheduled  for  extensive  testing  until  Spring  of  1981. 
A  computer  listing  of  all  DABS  trouble  reports  is  contained  in  Appen¬ 
dix  C. 
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3.  APPROACH  TO  SOFTWARE  RELIABILITY  MODELING. 


3.1  THEORETICAL  SOFTWARE  RELIABILITY  MODEL. 

The  reliability  growth  model  introduced  by  Duane  in  1964  and  more 
recently  expounded  by  Codier,  has  found  wide  acceptance  by  reliability 
engineers.  It  is  simple  to  use  and  it  is  applicable  to  both  continu¬ 
ous  and  discrete  data  cases.  Its  wide  applicability  to  diverse  hard¬ 
ware  test  programs  and  more  recently  to  software  test  data  prompted  its 
use  here. 

Using  data  from  several  different  types  of  hardware  test  programs  as  a 
basis,  Duane  plotted  cumulative  failure  rate  (Aj-)  vs.  total  operating 
time  (t)  and  observed  a  linear  relationship  between  log  (A^)  and  log 
(T)  for  each  equipment.  This  relationship  is  characterized  by  the 
model : 

A„  =  KT  a  where, 

A^  =  cumulative  failure  rate 

K  =  a  model  parameter  to  be  estimated  (represents  A  at  T=l) 

T  =  total  operating  hours,  cycles  or  missions 
a  =  Growth  rate  to  be  estimated. 

Duane  presents  a  method  for  estimating  the  model  parameter  directly 
from  the  data  plotted  on  log-log  paper.  The  growth  parameter  can  be 
obtained  by  calculating  the  slope  of  the  line.  The  location  parameter 
is  also  obtained  directly  from  the  plot  as  the  value  for  Av  at  T  =  1. 

K  can  be  interpreted  as  the  initial  or  zero-age  failure  rate.  For 
software,  it  is  a  function  of  program  complexity,  size,  its  maturity 
relative  to  the  state-of-the-art  and  other  variables. 

The  curve  is  more  sensitive  to  the  exponent  ex  than  to  K.  The  exponent 
reflects  the  intensity  with  which  reliability  improvement  is  pursued; 
it  nearly  always  lies  between  .2  and  .5,  the  average  being  close  to  .3. 


In  addition  to  information  regarding  cumulative  failure  rates,  the 
predicted  failure  rate  at  any  point  in  time;  i.e.,  instantaneous 
failure  rate  At,  can  also  be  estimated  from  the  following  equation 
where  F  =  total  failures  and  all  other  variables  have  been  previously 
defined. 


ST  ”  ST  (KT 


=  (l-cOKT-01  =  (  1-ot)  >.r 
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Thus,  program  progress  can  be  modeled  using  cumulative  information  and 
can  be  continuously  monitored  using  the  current  information. 


3,2  DATA  ANALYSIS . 

The  operating  time  and  error  data  base  for  each  software  module  was 
analyzed  to  provide  inputs  into  the  Duane  model.  Using  chargeable 
errors  and  test  time  the  cumulative  error  rate  X^  =  total  failures/ 
total  time  was  calculated  for  each  month  in  which  at  least  1  error 
was  reported.  The  data  were  plotted  in  accordance  with  the  Duane 
growth  curve  requirements  and  model  parameters  were  estimated  if 
growth  were  evident. 

The  modeling  process  generated  a  model  of  cumulative  error  rate, 

Xj-  =  KT-01,  and  a  model  of  instantaneous  error  rate,  Xt  =  (l-a)X^  . 

The  reciprocals  of  error  rates  are  the  MTBFs,  cumulative  and  instanta¬ 
neous  respectively.  In  addition,  MTBCF,  i.e.,  mean  time  between 
critical  (severity  class  1)  failures,  values  were  calculated  where 
applicable . 

For  the  modules  where  reliability  improvement  was  evident,  the  models 
were  used  as  predictive  tools  to  estimate  the  error  rate  during  future 
testing.  In  this  analysis  the  future  consisted  of  the  1790  test  hours 
during  July,  August  and  September  of  1980.  Results  of  the  predictions 
were  then  compared  to  actual  data.  The  analysis  of  the  COM  module 
(Section  4.1)  serves  as  a  detailed  example  of  the  process.  For  informa¬ 
tion  purposes,  Table  2  contains  a  listing  of  chargeable  errors  written 
during  the  prediction  test  interval. 
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Table  2.  Chargeable  Failures  Reported  During  the  Prediction  Interval 


Report 

Number 

Date  of 

Error 

Module 

Severity 

M0002 

7/10/80 

CM 

1 

N0066 

7/11/80 

MTD 

2 

S0317 

7/  16/S 

PM 

2 

S0319 

7/16/80 

PM 

2 

S0320 

7/16/80 

PM 

1 

S0321 

7/18/80 

SYS 

1 

S0326 

7/20/80 

COMM 

2 

S0328 

7/23/80 

SURV 

2 

SO  330 

7/24/80 

CM 

O 

S0331 

7/24/80 

SYS 

2 

S0333 

7/24/80 

SURV 

1 

N0072 

8/11/80 

SURV 

2 

S0334 

8/  4/80 

COMM 

2 

S0335 

8/  4/80 

COMM 

2 

S0337 

8/  4/80 

COMM 

2 

S0338 

8/  4/80 

COMM 

2 

M0004 

9/29/80 

SURV 

2 

M0003 

9/29/80 

SURV 

2 

M0006 

9/29/80 

SURV 

? 

M0008 

9/29/80 

SURV 

2 

MOO  14 

9/29/80 

l)EX 

2 

S0358 

9/15/80 

SYS 

2 
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4.  RELIABILITY  EVALUATION  OF  DABS  SOFTWARE  MODULES. 


4.1  RELIABILITY  EVALUATION  OF  THE  COMMUNICATION  MODULE . 

The  Communications  (COM)  Module  includes  the  surveillance  and  CIDIN 
communications  programs  which  control  and  monitor  transfer  of  data 
between  a  sensor  and  external  facilities.  The  reliabilities  of  other 
software  in  the  intersite  communications  package  were  not  modeled 
because  the  software  is  associated  with  off-line  and  maintenance  pro¬ 
cessing  and  is  not  part  of  the  mission  software. 

The  COM  module  was  tested  for  4966  hours  during  which  11  chargeable 
errors  were  reported  (see  Table  3).  Only  4091  test  hours  along  with 
the  11  errors  were  used  to  construct  the  growth  model  because  the 
data  base  was  terminated  at  the  last  reported  failure  in  accordance 
with  the  rules  of  model  construction.  Results  of  the  model  genera¬ 
tion  and  reliability  predictions  follow. 

The  model  which  describes  cumulative  error  rate  is  Xj;  =  .174 
where  X£  =  cumulative  error  rate  and  T  =  cumulative  test  hours.  For 
example,  at  the  time  of  the  last  reported  error  (T  =  4091  hr.)  the 
model  predicts  X^  =  .00265  error/hr.  or  377  hr.  MTBF.  The  measured 
error  rate  at  T  =  4091  hr.  was  .0027  error/  hr.  or  372  hr.  MTBF.  When 
used  as  a  predictive  tool  to  extrapolate  beyond  the  time  limits  of 
the  data  base  to  T  =  6756  hours,  Xjr  =  .174  (6756)~*503  =  .00206  error/ 
hr.  or  485  hours  MTBF.  The  time  interval  between  4091  hours  and  6756 
hours  (2665  hours)  includes  the  last  875  hours  of  test  without  a 
reported  failure  and  1790  hours  of  test  during  the  prediction  interval. 
Because  the  model  predicted  a  cumulative  error  rate  of  .00206  error/hr. 
at  T  =  6756  hr.,  the  expected  number  of  cumulative  errors  was  calcu¬ 
lated  from:  F  =  Xj •  T  =  .00206  error/hr.  •  6756  hr.  =  135  errors. 
Because  11  errors  had  already  been  reported  within  4091  test  hours, 
the  remaining  2.9  errors  represent  a  prediction  to  be  compared  with 
the  observed  results.  Tn  fact,  five  chargeable  errors  were  reported 
against  the  COM  module,  a  number  which  is  well  within  the  limits  of 
statistical  variation.  Figure  4  contains  a  graph  of  the  model. 

The  model  which  describes  instantaneous  error  rate  and  MTBF  is 
Xt  =  .0865  T-'-’®^.  It  represents  the  rate  at  which  errors  are  system¬ 
atically  being  identified  and  removed  from  the  COM  module  at  time  T 
hours.  For  example,  at  T  =  6756  hours,  Xt  =  .0865  (6756)-'-’®^  = 

.00102  error/hr.  or  976  hr.  MTBF.  The  value  of  Xt  at  T  =  6756  has 
the  significance  that  if  the  test  correction  process  were  to  cease  at 
T  =  6756  hours,  the  error  rate  of  the  COM  module  would  no  longer 
decrease,  but  u’ould  become  constant  at  X  =  .00102  error/hr.  or  976  hr. 
MTBF. 
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Table  3.  Communication  Module  -  Reliability  Data  Summary 


Month 

Monthly 
Test  Hrs 
(T) 

Monthly 

Errors 

(F) 

Cumulative 
Test  Hrs. 

<V 

Cumulative 

Errors 

<V 

Cumulat ive 
Er ror/Hr 

<v 

Average 

Time 

Between 

Er  rors 
(MTHFj.) 

1979  Feb 

Mar 

Apr 

May 

140 

1 

140 

1 

.0071 

140 

June 

140 

1 

280 

2 

.0071 

140 

July 

188 

1 

468 

3 

.0064 

156 

Aug 

188 

1 

656 

4 

.0061 

1  64 

Sopt 

188 

1 

844 

5 

.0059 

, 

169 

Oct 

305 

0 

1149 

1  _ 

Nov 

390 

0 

1539 

1979  Dec 

489 

0 

2028 

1980  Jan 

528 

1 

2556 

6 

.0023 

4  26 

Feb 

480 

3 

3036 

9 

.003 

337 

Mar 

431 

1 

3467 

10 

.0029 

347 

Apr 

624 

1 

4091 

11 

.0027 

372 

May 

418 

4509 

June 

457 

4966 

J  u  ly 

Aug 

Sept 

1790 

6756 

18 


00  1,000  10,000 

Cumulative  Test  Hours 


Figure  4.  Communication  Module  -  Reliability  Growth  Model 
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4 . 2_  RELIABILITY  EVALUATION  OF  THE  PERFORMANCE  MONITORING  MODULE . 

The  Performance  Monitoring  (PM)  module,  a  portion  of  intrasite  commu¬ 
nications,  is  responsible  for  gathering  and  analyzing  status  of  the 
sensor  and  the  transmission  of  status  messages  to  external  facilities. 

As  reported  in  Table  4,  the  PM  software  was  tested  for  4966  hours 
during  which  20  chargeable  errors  were  reported.  Of  the  chargeable 
errors,  five  were  classified  as  critical.  Figure  5,  which  contains 
the  reliability  growth  curve  for  the  PM  module  shows  that  the  module 
experienced  a  decreasing  error  rate  throughout  the  test  except  for 
minor  fluctuations.  The  cumulative  error  rate  model,  =  .  1403T~-^^, 
gives  Xj  =  .00348  error/hr.  at  T  =  6756  hours.  This  translates  into 
3.5  expected  errors  during  the  time  of  the  test  interval  used  for  pre¬ 
diction  purposes.  Because  3  chargeable  errors  were  reported  during 
the  1790  test  hours  of  the  prediction  interval,  there  is  close  agree¬ 
ment  and  acceptance  of  the  model. 

-.419 

The  instantaneous  error  rate  model,  Xt  =  .0815T  '  ,  predicts  that 

X  =  .00203  error/hr.  at  T  =  6756  which  is  equivalent  to  494  hours  MTBF. 

4.3  _RE_L  I  ABILITY  EVALUATION  OF  MESSAGE  ROUTING  AND  DATA  LINK  PROCESSING 

mo'dulks'.  . ~ . . 

'Hie  Message  Routing  (MR)  software  is  responsible  for  routing  incoming 
messages  to  the  appropriate  software  module.  Data  Link  (DL)  processing 
manages  uplink  and  downlink  messages  to/from  participating  DABS 
equipped  aircraft.  MR  &  DL  software  were  tested  together  and  form  a 
single  software  module  for  the  purpose  of ' reliability  analysis. 

Table  5  contains  the  time  and  error  data  used  to  generate  the  relia¬ 
bility  growth  curve  shown  in  Figure  6.  Using  the  cumulative  growth 
model,  Xj-  =  .3467  t-645^  ^  =  ,00117  error/hr.  at  T  =  6756  hours. 

The  model  predicts  the  occurrence  of  7.9  errors  throughout  the  test 
of  6756  hours.  Because  7  errors  were  reported  during  the  initial 
test  phases,  0.9  errors  were  predicted  to  occur  during  the  prediction 
test  interval.  Actually,  no  failures  were  reported  during  the  1790 
hours  of  additional  test,  a  value  within  acceptable  statistical  vari- 
at  ion . 

- .  645 

The  instantaneous  error  rate  model,  Xt  =  .123T  ’  ,  predicts  Xt  = 

.000417  error/hour  or  2400  hours  MTBF  at  T  =  6756  hours. 
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Table  4 .  Performance  Monitoring  Module  -  Reliability  Data  Summary 


Month 

Monthly 
Test  Hrs 
(T) 

Monthly 

Errors 

(F) 

Cumulative 

Test  Hrs. 

<V 

Cumulat ive 
Errors 

<f£) 

Cumul at ive 
Error /Hr 

<V 

1979  Feb 

_ 

' 

Mar 

Apr 

. . . 

. 

May 

140 

3 

140 

3 

.0214 

June 

140 

0 

280 

July 

188 

2 

468 

5 

.0107 

Aug 

188 

0 

656 

Sept 

188 

1 

844 

6 

.0071 

0c  t 

305 

5 

1149 

11 

.  0096 

Nov 

390 

0 

1539 

1979  Dec 

489 

1 

2028 

12 

.0059 

1980  Jan 

528 

1 

2  5  56 

13 

.0051 

Feb 

480 

3036 

Mar 

431 

3467 

Apr 

624 

4 

4091 

17 

.0042 

May 

418 

2 

4  509 

19 

.0042 

June 

457 

i 

4966 

20 

.0040 

July 

Aug 

1790 

Sept 

6756 

Average 
T  i  me 
Between 
Errors 
(MTBF.O 


A  7 


93 

141 


10A 


169 


196 


2  38 
2  38 
2  60 


-ror/Hr) 


Table  5.  Message  Routing  and  Data  Link  Modules  -  Reliability  Data  Summary 


Month 

Monthly 
Test  Hrs 
(T) 

Monthly 

Errors 

(F) 

1 

1 

Cumulative 
Test  Hrs. 

<V 

! 

Cumulative 

Errors 

<Fi> 

Cumulat ive 
Error/Hr 

<v 

1979  Feb 

j"  ‘  ~~  " 

. 

- -  _  . 

— 

Mar 

. 

Apr 

1 . ' 

May 

140 

1 

140 

1 

.0071 

June 

140 

280 

July 

188 

2 

468 

3 

.0064 

Aug 

188 

656 

Sept 

188 

1 

844 

4 

.0047 

Oct 

305 

. - 

1149 

'  - 

Nov 

390 

1539 

1979  Dec 

489 

2028 

1980  Jan 

528 

‘  *  -  - - 

2556 

• -  ’  '  • 

Feb 

480 

3036 

Mar 

431 

2 

3467 

6 

.0017 

Apr 

624 

1 

4091 

7 

.0017 

May 

418 

4509 

June 

457 

4966 

July 

Aug 

1790 

Sept 

6756 

Average 

Time 

Between 

Errors 

(MTBFj,) 


141 


156 


213 


5  88 


588 


VC 


A. A  RELIABILITY  EVALUATION  OF  DATA  EXTRACTION  MODULE. 


Included  in  this  evaluation  are  data  from  the  portion  of  the  Data 
Extraction  (DEX)  module  which  is  associated  with  data  collection;  i.e., 
on-line  real  time  extraction  of  performance  data  from  the  DABS  data 
base  and  its  recording  on  magnetic  tape.  Playback,  quick-look  and 
extended  analysis  software  are  used  off-line  and  are  not  part  of 
the  mission  software. 

As  seen  in  Table  6  and  Figure  7,  the  DEX  module  is  very  reliable.  Only 
two  chargeable  errors  were  reported  in  5386  test  hours  for  an  MTBF  of 
2693  hours  or  an  error  rate  of  .000371  error/hr.  One  of  the  errors 
was  classified  as  critical.  Too  few  data  are  available  to  generate  a 
reliability  trend  curve.  It  can  be  seen  however,  that  the  error  rate 
is  decreasing  which  implies  that  the  instantaneous  MTBF  is  greater 
than  2693  hours.  There  was  one  error  reported  against  DEX  software 
during  the  prediction  test  interval. 

A. 5  RELIABILITY  EVALUATION  OF  CHANNEL  MAN ACE MEN T  MO PULE . 

Channel  Management  (CM)  regulates  all  activity  on  the  RF  channel, 
scheduling  the  aircraft  interrogations  and  corresponding  listening 
periods  to  ensure  that  communications  and  surveillance  tasks  are 
accomplished  for  each  aircraft. 

The  CM  module  has  been  characterized  by  T.  I.  software  designers  as 
the  most  complex  of  the  DABS  software  modules  primarily  because  of  its 
logical  structure.  Its  measured  reliability  is  among  the  lowest. 

During  5386  hours  of  test,  1A  chargeable  errors  were  reported  for  an 
error  rate  of  .0026  error/iir.  or  385  hours  MTBF.  Table  7  contains 
cumulative  time  and  error  data  for  CM.  The  data  plot  in  Figure  8 
shows  that  no  trend  analysis  is  possible  because  of  the  abrupt  changes 
in  slope  of  the  curve.  In  fact,  during  the  test  period  between  2500 
hr.  and  5000  hr.  CM  error  rate  increased  from  .0016  error/hr.  to 
.0028  error/hr.  Subsequent  testing  indicates  a  reversal  of  the  trend 
because  the  error  rate  appears  to  be  decreasing  in  the  prediction 
interval  between  A929  hours  and  7176  hours.  Consequently,  for  predic¬ 
tion  purposes  the  cumulative  error  rate  A-  =  total  errors/total  hours  = 
.0026  error/hr.  will  be  used. 


A.  6  REL  I_AB  I L 1TY  EVALUATION  OF  NETWORK  MANACEMENT  MODULE. 

Network  Management  (NM)  is  a  portion  of  intrasite  communications 
responsible  for  communication  of  surveillance  data  to  and  from  other 
sensors . 

Table  8  contains  the  data  used  in  the  reliability  analysis  of  the  NM 
software.  Based  on  a  total  of  A 122  test  hours  and  22  chargeable  errors 


Table  6.  Data  Extraction  Module  -  Reliability  Data  Summary 
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Table  7.  Channel  Management  Module  -  Reliability  Data  Summary 
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Table  8.  Network  Management  -  Reliability  Data  Summary 


Month 
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1979  Dec 


1980  Jan 
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NM  cumulative  error  rate  was  measured  to  be  Xj;  =  22/4122  =  .00534 
error/hr.  or  187  hours  MTBF.  NM  software  has  been  characterized  as 
moderately  complex;  it  has  the  highest  predicted  error  rate  of  the 
DABS  software  modules  which  were  studied.  Although  its  error  rate 
appears  to  be  decreasing  (see  Figure  9),  there  are  not  sufficient  cur 
rent  data  to  support  trend  analysis.  Data  reported  during  the  pre¬ 
diction  test  interval  also  indicate  a  decreasing  error  rate  trend. 
During  1790  hours  of  test  there  were  no  reported  errors.  However, 
a  large  portion  of  the  apparent  reliability  improvement  may  be  due 
to  a  lessening  of  the  severity  of  the  test  environment.  The  formal 
NM  test  which  was  run  to  demonstrate  compliance  with  operational 
requirements  had  been  completed  in  dune  1980.  Consequently  the  NM 
software  may  have  been  operating  in  reduced  data  and  requirements 
environments  during  the  July  to  September  time  frame. 

4._7  RELIABILITY  EVALUATION  OF  MTD  AND  J1DAS_  MODULES. 

The  Moving  Target  Detector  (MTD)  and  Radar  Data  Acquisition  Subsystem 
(RDAS)  programs  are  integral  to  the  sensor  track  software  which  is 
required  to  acquire  and  track  DABS  and  ATCRBS  aircraft. 

As  noted  in  Table  9,  the  MTD  and  RDAS  module  was  tested  for  only  3 
months  resulting  in  1499  test  hours  and  3  chargeable  errors  for  an 
error  rate  of  .002  error/hr.  or  500  hours  MTBF.  The  data  in  Figure 
10  show  a  constant  error  rate,  with  a  slight  increasing  trend  which 
is  influenced  by  the  paucity  of  data.  During  the  prediction  test 
interval  no  errors  were  reported  against  the  MTD  and  RDAS  module. 


4^ 8  RELIABILITY  EVALUATION  OF  SYSTEM  SOFTWARE  MODULE. 

System  (SYS)  software  refers  to  all  the  software  required  to  calibrat 
and  initialize  DABS  and  recover  from  hardware  failures.  SYS  also  con¬ 
tains  standardized  software  support  utilities. 
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Table  10.  System  Software  Module  -  Reliability  Data  Summary 


Month 
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prediction  that  one  error  will  occur  during  the  prediction  test  inter¬ 
val.  Three  chargeable  errors  were  reported,  a  number  which  is  quite 
high.  The  occurrence  of  three  or  more  errors  when  only  one  is  expected 
should  occur  no  more  than  8  percent  of  the  time.  Therefore,  the 
prediction  is  marginally  acceptable. 

The  instantaneous  error  rate  of  .000386  error/hr.  at  T  =  6756  hours 
was  predicted  using  the  model  A  =  .  78T-'®^.  Based  on  the  above 
comparison  between  predicted  and  actual  numbers  of  failures,  the 
instantaneous  failure  rate  is  expected  to  be  somewhat  optimistic. 

4.9  RELIABILITY  EVALUATION  OF  THE  SURVEILLANCE  PROCESSING  MODULE. 

The  Surveillance  (SURV)  processing  module  is  responsible  for  tracking 
targets,  correlating  radar  reports  with  beacon  reports  or  tracks  and 
for  maintaining  the  surveillance  file. 

The  SURV  module  accrued  5386  hours  of  test  during  which  41  chargeable 
errors  were  reported.  Its  cumulative  error  rate  at  T  =  5386  was  .0076 
error/hr.  or  131  hours  MTBF.  The  data  used  in  the  reliability  anal¬ 
ysis  is  contained  in  Table  11.  Figure  12  displays  the  reliability 
growth  model.  It  should  be  noted  that  the  SURV  module  exhibited 
decreasing  error  rates,  but  at  different  rates.  The  change  in  slope  of 
the  curve  may  be  attributed  to  variations  in  the  test  environment,  to 
delays  in  documenting  the  errors  or  to  delays  in  implementing  correc¬ 
tive  action.  All  three  situations  have  been  identified  as  causative 
factors  which  perturb  reliability  growth  models.  Note  that  the 
growth  curve  was  generated  using  weighted  least  squares  which  stresses 
current  data.  The  slope  of  the  line  favors  the  current  trend  rather 
than  the  overall  trend. 

Based  on  the  cumulative  growth  model,  Aj;  =  .  1067T  ,  the  predicted 

A;r  at  T  =  7176  hours  is  .006983  error/hr.  which  is  equivalent  to  a 
total  of  50.1  expected  errors.  This  implies  a  prediction  of  9.1 
errors  during  the  1790-hour  prediction  test  interval.  Seven  (7) 
errors  were  reported  against  the  SURV  module,  a  number  which  compares 
favorably  with  the  prediction. 

Hie  instantaneous  growth  model,  At  =  .0739T  predicts  an  error 

rate  of  .00484  error/hr.  or  207  hours  MTBF  at  T  =  7176  test  hours. 
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5.  INTEGRATED  DABS  HARDWARE  AND  SOFTWARE  RELIABILITY  MODEL. 


5.1  SOFTWARE  SUBSYSTEM  RELIABILITY  MODEL. 

The  reliability  block  diagram  of  the  DABS  software  is 
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model  which  corresponds  to  the  above  block  diagram  is 


Vw.  "  TT  Ri 

i=l 


where  R^  ^  =  Reliability  of  the  software  subsystem 

R^  =  Reliability  of  each  module,  i=l,  9 

It  was  noted  in  Section  4  of  this  report  that  the  reliability  growth 
model  used  to  measure  DABS  software  reverts  to  a  constant  failure  rate 
model  at  the  conclusion  of  the  test/analyze/fix  process  underway  during 
a  test  program.  Therefore,  the  reliability  model  for  operational 
software  is  identical  to  the  reliability  model  used  by  the  FAA  to 
model  hardware  reliability.  It  is  characterized  by  an  exponential 
distribution  of  times  between  errors  or  it  may  be  stated  as  a  Poisson 
distribution  of  the  number  of  errors  within  a  specified  time  interval. 

Using  terminology  similar  to  that  used  by  the  FAA  in  the  hardware 
reliability  model, 


S.W. 


N 
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where  W  =  error  rate  of  the  total  software  program 

X.  =  error  rate  of  a  software  module  for  i= 1 ,  9. 

As  noted  earlier,  the  error  rates  of  the  modules  are  changing  because 
of  the  results  of  debugging  the  software.  Therefore,  the  reliability 
prediction  will  be  made  in  terms  of  accumulated  software  test  time. 

A  summary  of  reliability  predictions  for  the  DABS  software  modules  is 
presented  in  Table  12.  In  summary,  it  states  that  a  chargeable  error 
will  occur  within  the  DABS  software  every  53  test  hours.  For  compari¬ 
son  purposes.  Table  13  contains  a  summary  of  module  critical  error 
rates . 

It  was  noted  earlier  that  the  error  rate  of  a  module  may  improve 
dramatically  once  the  module  is  removed  from  a  test  environment.  An 
improvement  factor  of  5  was  noted  by  the  author  in  a  similar  study. 

It  is  not  implied  that  the  same  factor  is  applicable  to  DABS  software, 
but  if  it  were,  the  time  between  chargeable  errors  would  increase  to 
only  265  hours. 

The  software  reliability  model  makes  no  provision  for  software  repair 
in  the  event  of  failure.  The  DABS  system  is  structured  to  provide 
reconfiguration  in  the  event  of  certain  hardware  failures.  Critical 
software  failures  will  generally  fail  the  system.  Also,  redundancy 
features  of  hardware  do  not  apply  to  software.  If  two  redundant  pro¬ 
cessors  encounter  the  same  logical  software  error,  and  if  the  error  is 
critical,  both  processors  and  therefore  the  computer  will  fail. 


5.2  DABS  INTEGRATED  SYSTEM  (HARDWARE  AND  SOFTWARE)  RELIABILITY  MODEL. 

Hie  reliability  block  diagram  which  combines  hardware  with  software 
elements  of  DABS  is 


Hardware  ! 
Elements  j 


Sof  tware  I 
Elements  I 


The  reliability  model  is  x  Rg>w>  =  R^g. 

Translated  into  the  effective  failure  rate  (X^pp)  model  used  by  FAA, 
''dABS  =  ^£pp(Hardware)  +  X(Software).  Based  on  data  contained  in'  Re¬ 
port  No.  FAA-RD-80-36 ,  "Discrete  Address  Beacon  System  (DBAS)  Base¬ 
line  Test  and  Evaluation",  by  M.  Holtz,  Xppp  =  .00  1736  failure/hr. 
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Table  12.  Summary  of  Software  Reliability  Predictions 

Reliability  Predictions 


Software  Module 

Number  of  Errors 
Within  Prediction 
Interval 

Number  of 
Errors/ 
Test  Hour 

Average  Time 
Between 
Errors 

Communications 

2.9 

.001020 

976 

Performance  Monitor 

3.5 

.002030 

494 

Message  Routing  & 

Data  Link 

0.9 

.000417 

2400 

System  Software 

1 

.000386 

2588 

Surveillance  Processing 

9. 1 

.004840 

207 

Data  Extraction 

No 

Predi ction 

.00037 

2703 

Channel  Management 

No 

P  re  d  i  c  t  i  on 

.00260 

385 

Network  Management 

No 

P re die  tion 

.00534 

187 

MTD  &  RDAS 

No 

Predi ct ion 

.00200 

500 

TOTAL 

.019 

53 
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Table  13.  Summary  of  Module  Critical  Error  Rates 


Software  Module 

Test 

Hours 

Number  of 
Critical 
Errors 

Critical  Error 
Rate  -  Critical 
Error/Hr. 

COM 

4966 

5 

.001 

PM 

4966 

5 

.001 

MR  &  DL 

4966 

0 

— 

DEX 

5  386 

1 

.00019 

CM 

5386 

4 

.00074 

NM 

4122 

2 

.000485 

MTD  &  RDAS 

1499 

1 

.000667 

SYS 

4966 

7 

.0014 

SURV 

5  386 

6 

.00111 

.00662 


Note:  During  July,  August  and  September  of  1980,  4  critical  errors 

were  reported  during  1790  hours  of  test.  The  error  rate  of 
.00223  error/hr  is  equivalent  to  MTBCF  of  448  hours. 
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.001736  +  .019  = 


Using  all  chargeable  software  errors, 

.020736  failure/hr.  or  48  hours  MTBF.  Using  only  critical  software 
errors,  X_.__  =  .001736  +  .00223  =  .003966  fail/hr.  or  252  hours  MTBF. 

DAJ5  o 
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APPENDIX  A 


Review  and  Critique  of  the  Available 
Hardware  Reliability  Model  and  the  Hardware 
Reliability  Prediction  for  the  DABS 


FAA  Report  No.  FAA-NA-78-31,  "Plan  for  the  Reliability  and  Main¬ 
tainability  Evaluation  of  the  Discrete  Address  Beacon  (DABS)  Engineer¬ 
ing  Laboratory  Models,"  contains  the  hardware  reliability  model  and 
reliability  prediction  for  the  DABS.  The  report  also  addresses  the 
failure  reporting,  data  collection,  data  processing  system  and  the 
criteria  which  will  be  used  to  evaluate  (measure)  hardware  reliability 
of  engineering  laboratory  models. 

The  critique  presented  herein  addresses  only  those  portions  of 
the  report  which  deal  with  the  DABS  hardware  reliability  models  and 
the  reliability  prediction.  Review  and  critique  of  the  failure  data 
collection,  processing  and  analysis  procedures  are  outside  the  scope 
of  this  task. 

The  FAA  report  describes  the  construction  and  use  of  a  series  of 
Einhorn  equations  (models)  which  transform  mean-time-between-failure 
(MTBF)  and  mean-t ime-to-repair  (MTTR)  of  an  equipment  into  effective 
failure  rate  (^ppp)  for  the  equipment.  In  addition,  a  method  is  pro¬ 
vided  by  which  effective  failure  rates  of  2  or  more  equipments  may  be 
combined  to  produce  a  subsystem  effective  failure  rate.  The  models 
of  the  DABS  subsystems  are  well  prepared  and  documented.  The  comments 
which  follow  address  minor  points  of  the  modeling  process  and  several 
major  topics  which  were  not  addressed  in  the  report,  but  which  might 
be  candidates  for  inclusion  in  a  revision  to  the  document. 

GENERAL  COMMENTS : 

1.  The  predicted  mean-t ime-between-failures  (MTBF)  of  a  single 
channel  sensor  is  774  hours,  a  value  considerably  lower  than  the  1,000 
hours  specified  in  the  engineering  requirement.  There  is  nothing  in 
the  document  to  indicate  that  appropriate  improvements  such  as 
redesign  or  use  of  high  reliability  parts  will  be  employed  to  improve 
system  reliability.  As  stated  in  Report  No.  FAA-RD-80-36,  DABS 
measured  MTBF  is  575  hours  but  is  increasing.  The  FAA  should  monitor 
test  results  closely,  continue  to  measure  system  MTBF  during  develop¬ 
ment  testing  and  implement  effective  corrective  actions  to  improve 
system  MTBF  to  meet  the  specification. 

2.  The  state  diagram  technique  used  to  model  DABS  hardware  reliability 
appears  to  generate  ^Epp  which  is  pessimistic.  Significant  terms  in 
the  calculation  of  ^Epp  are  obtained  by  multiplying  the  failure  rate 

of  a  specific  hardware  configuration  by  the  probability  of  failure  in 
the  configuration  for  the  entire  anticipated  mission.  However,  the 
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most  likely  time  of  failure  for  equipments  having  MTBF  >>  mission  time 
is  near  the  midpoint  of  the  mission.  Hence,  the  calculated  probabilities 
of  failure  are  nearly  doubled. 

3.  The  report  should  contain  a  brief  but  complete  description  of  the 
DABS  mission.  A  complete  reliability  evaluation  plan  should  describe 
the  anticipated  mission  or  a  standardized  mission  which  will  be  used 
for  reliability  measurement.  Mission  identification  should  identify 
and  describe  all  mission  phases,  their  duration  and  anticipated 
environments.  The  results  of  the  mission  analysis  should  then  be  merged 
with  the  results  of  a  systems  analysis  which  then  identifies  the  full 
complement  of  equipment,  including  reliability  block  diagrams,  which 
will  be  used  to  measure  reliability  during  each  mission  phase.  Also 
included  are  alternate  modes  of  operation  and  success/failure  criteria. 

4.  The  reliability  equations  are  very  general  and  optimistic  because 
they  include  the  probability  of  repairing  equipments  without  consider¬ 
ing  the  number  of  repairmen,  number  of  spares  or  administrative  delays 
which  may  prolong  maintenance  time.  The  FAA  equations  are  applicable 
only  if  an  infinite  number  of  spares  and  repairmen  are  instantly 
available  at  each  operational  site.  If  reasonable  constraints  were 
placed  on  the  above  model  parameters,  predicted  reliability  would 
decrease. 

5.  The  FAA  report  specifically  states  that  "special  reliability  tests" 
will  not  be  conducted  and  that  objectives  of  the  reliability  and 
maintainability  (R&M)  evaluation  can  be  achieved  using  FAA  performance 
tests.  It  should  be  recognized  that  not  all  performance  tests  will  be 
applicable  to  the  measurement  of  DABS  R&M.  The  report  should  contain 
the  results  of  an  analysis  of  the  anticipated  test  program  which  would 
describe  the  quantity  and  quality  of  the  anticipated  data  and  why  the 
data  can  be  used  for  R&M  measurement. 

6.  The  "estimation"  of  equipment  MTBF  should  include  the  calculation 
of  confidence  intervals  for  equipment  and  system  MTBF;  i.e.,  an 
interval  which  contains  the  true  but  unknown  MTBF  with  stated 
probability. 


S P ECIF IC  COMMENTS : 

1.  Rage  28  Second  Paragraph 

Per  this  paragraph.  A  statistical  test  will  be  performed  to 
determine  if  the  exponential  distribution  is  appropriate  to  describe 
time  to  failure  and  time  to  repair  data.  The  report  should  describe 
alternative  statistical  techniques  for  data  analysis  if  the  exponential 
distribution  fails  to  adequately  describe  these  time  data.  This  is 
especially  important  for  time  to  repair  which  is  often  modeled  using 
the  log-normal  distribution. 


2.  Pages  39/40 


The  reliability  block  diagram  for  two  redundant  equipments  with 
a  switch  is 


The  reliability  model  for  the  above  system  is 

R  =  6  +  rswitch  (At)  6 

where  X  =  failure  rate  of  176  K  memory  string 

t  =  mission  duration 

R  =  reliability  of  switching  element 

bW I I Ln 

3.  Titles  of  Figures  1,  2,  4  and  12 

These  figures  are  titled  "reliability  models"  but  a  more 
appropriate  title  is  "reliability  block  diagram."  The  reliability 
model  is  usually  defined  as  the  equation  which  transforms  MTBF  into 
probability  of  success. 


APPENDIX  B 


A  Recommended  Software  Reliability  Failure 
Reporting  System  for  DABS 


The  FAA  currently  uses  the  DABS  Trouble  Report/Change  Proposal 
(Figure  B-l)  to  document  software  errors.  Additional  analysis  and 
follow-up  are  documented  on  DABS  Trouble  Report/Change  Proposal  Update 
Worksheet  (Figure  B-2).  While  the  reporting  system  was  not  structured 
specifically  to  provide  data  suited  for  reliability  analysis,  the 
forms  do  provide  most  of  the  required  information  when  completed  in 
accordance  with  the  Trouble  Report  Users  Guide. 

The  difficulties  encountered  when  using  DABS  Trouble  Reports  (TR) 
were  primarily  lack  of  completeness  and  lack  of  error  classification. 
These  weaknesses  in  the  present  system  can  be  corrected  by  instituting 
an  editorial  review  of  the  software  errors  as  TRs  are  initiated,  com¬ 
pleted  and  classified.  A  glaring  weakness  in  the  procedure  can  be  cor¬ 
rected  by  ensuring  that  the  TRs  contain  the  date  of  occurrence  of  the 
error  as  well  as  the  date  of  TR  initiation.  It  is  recommended  that  a 
representative  of  the  reliability  engineering  group  participate  in  the 
editorial  review  because  much  of  the  data  are  needed  for  reliability 
analysis  purposes'. 

As  soon  as  error  follow-up  identifies  causes  for  the  initiation 


of  a  TR  it  should  be  classified  with  regard  to  CATEGORY  and  SEVERITY. 
With  regards  to  CATEGORY,  the  following  definitions  are  recommended 
for  use  by  FAA. 

Error  Source 
Code 

Error  Source 

Description 

0 

Req  ui  renents 

Source  of  problem  is  changing, 
ill  conceived  or  poorly  stated 
performance  requirement. 

1 

Design 

Source  of  problem  is  in  prelim¬ 
inary  or  detailed  design. 

2. 

Coding 

Source  of  problem  is  an  error  in 
implementing  the  design  or  code. 

3. 

Maintenance 

Source  of  problem  is  an  error  in¬ 
troduced  in  process  of  trying  to 
fix  a  previous  error. 

4. 

Not  Known 

Source  of  error  not  known. 
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E  OR  ORIGINATOR  US  C 


originator 

SOE I*ARE 

PRODUCT 
mr  Hunt  R 

part  number 

priority 
[]  critical 


ORlGlNA  TING 
INSTAL  L  ATION 


V  E  R  S  ION 
R  f  VISION 


DATE 

HO  J  DA  |  VI 
C  MANGE 

PROPOSAL  C  1 


1  ROUBLE 
HC  PORT 


i  ]  VERY  [3  IMPORTANT 

IMPORTANT 


[  1  c  auses 

incohve  mr  nce 


[  ]  INTERESTING 


GINERAL  DISCRETION 


ChICH  APPLICABLE  BOXES 

(  ]  A1C  INHRUCE  AMKTED 
t  ]  l  ISTING  A  I  TACMEO 

n  INST  A  L  L  A  I  ION  S  YST  l  M  PAT  c  HE  D  NO.  = 

[]  Tl  MPORARY  E CM  INSTAI  LED 
[  ]  A  I  TACMMF NTS 
[]  HAP  D  MARE  PART  FAIIURE 
[]  OTHER 

fOR  INVESTIGATING  ACTIVITY  ONLY 

PRGPQS  E  0  'Ol  U  TlON/COMME N 7ARY 


PP'ORlTY 

ASSiGNPO  ACTIVITY 

l 

,  T  ROUBLE  HPT  NO- 
C  MANGE  PROP.  NO. 
j  COMMENTS 


FOR  1C  C  B  USE  ONLY 
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Figure  B-l.  DABC  Trouble  Report/Cbange  Proposal 
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DAOS  TROUBLE  RCI'ORT/ChANCE  PROPOSAL  UPDATE  WORKSHEET 

INIIR  AT  l  f  AST  ONE  OF  THE  F  OL  t  "INC  TO  IOI  NTlF  V  RE  PORT 

HT  PORT  CHANCE 

i  ohm  no.  _  _  proposal  no 

SHORT  Ol  SCRIPT  ION 
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TROUBLE 
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INST  Al  IATIONS  AFFECTED 


OT  MLR  FORMS  INITIATED 

€CN  OCN 


CHANGE  TEST  RESUL  TS 

f  ?  Sw  C  HANGF  PROPOSE  0  DOTS  NOT  CORRECT  PROBLEM 
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l]  HR  CORRECTION  DOES  NOT  SOLVE  PROBLEM 

[]  HA  FIX  TEbTFD  AND  PEAOy  FOR  iHPl  I  “IMATlON  if  CN  ATTACHED) 

■  ROUBLE  CATE  GORY 

T]  H 'HARDWARE  PROBLEM 
[  ]  S  SOF  T  WARE  PROBLEM 
[J  O-  DOCUMENTATION  PROBLEM 
[  ]  C  -DE  SIGN  PROBl  EM 
Cl  R-'NE*  REQUIREMENT 

M  FORM  63tS-3fT  79) 


Figure  B-2.  DABS  Trouble  Report/Change  Proposal 
Update  Worksheet 


Errors  should  also  be  classified  as  to  type: 


Error  Type  Code 


Type  of  Software  Errors 


A 

B 

C 

D 

E 

F 

G 

H 

I 

J 

K 

X 


Computational 
Logic  Errors 
Data  Input  Errors 
Data  Handling  Errors 
Data  Output  Errors 
Interface  Errors 
Data  Definition  Errors 
Data  Base  Errors 
Operational  Errors 
Other 

Documentation  Errors 
Trouble  Report  Rejection 


Trouble  reports  should  be  classified  as  with  regard  to  SEVERITY  in 
accord  with  the  following  definitions: 

Chargeable  Errors: 

1.  Critical  -  An  error  in  the  software  which  causes  a 
significant  or  total  loss  of  operational  system  capability. 

2.  Non-Critical  (Major)  -  An  error  in  the  software  which 
causes  an  erroneous  response  in  the  operational  system.  An  error 
in  this  classification  may  not  be  recognized  as  such  by  a  trained 
observer  due  to  the  self  repair  inherent  in  the  system. 

Non-Cha rgeable  Errors: 

3.  Other  (Minor)  -  An  error  which  has  no  measurable  ef¬ 
fect  on  the  operational  system  or  are  of  unkown  cause  at  this  time 
(hardware/software/cockpit).  Errors  of  unknown  cause  would  be  charged 
against  the  DABS  system  rather  than  the  software. 

4.  No  Count  -  A  trouble  report  which  was  erroneously  attributed 
to  software  errors.  In  addition,  change  proposals,  enhancements, 
duplicate  trouble  reports  and  "cockpit  errors"  are  included. 
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1.0161  DS-S-0 124 -0048  25NM  2-1  TRANS  III  ON  DAB007  NM  ##12/1*  79  II 

SO  1 63  -  SNS  DABS  TARGETS  DU1SIDE  COV  DAE007  NM  *#12/1*  79  l 

90184  DS-S -01 ,18-0067  16CHANCED  LCF  FJ1ES  DAB007  -  ##0l/2.  80  11 

SO  1 4  9  DS  S -  0 1 3 1 -0067  11  INCORRECT  AC  TABS  REPLY  A  EL  DAB007  SURV  #*12/0*  79  11 

b0179  DS  S-01 37  -0067  1 5NUMDER  OE  CONN  SCNi.ORS  =  0  DAB007  PM  *»0l/.:  80  11 

90169  DS  -S-0135  0067  MALL  CALL- TO-COAST  DATA  REQU  DAS007  NM  *#Ol/C'  SO  11 

SOI  75  DS  S-01 36-0046  3 1  DAT  A  STOP  U1TH  ZERO  ID  CAB007  NM  #»01/1.  80  11 

SOJ80  DS-S -01 40-0049  33REQ  FOR  PRIM  IN  CENT  CECLDA2007  NM  *01/1  SOS  5S 

SO  174  DS-S-0139-0048  32  NET  MG T  HI  CON  1EST  DAB007  NI1  #«01/0:  80  3 

SO  167  DS  S-01 33-0067  13  MULTISITE  ADAPTATION  DAE007  SA  *#01/0'  60  11 

SO  1 65  DS  S-0141  0048  34  OVERLOAD  COMM  t<  DaaT  FACIL  DAB007  NM  ##12/2!  79  11 

90166  DS -S-01 34  0048  30  CHFCKS  FOR  UNLK  LOCKED  F I L F DPB007  NM  **12/2!  79  11 

E0164  DS-S-01S2-0067  12  LOST  DABS  DUE  TO  DAAT  DAB007  NM  *#12/3.  79  11 

NG004  DS-S-0130  -  0078  00  InCOR  ROLL-CALL  REPLIES  DAB006  CM  ##12/2;  79  11 

soiee  ds-s-oi 54-0009  oo  atcrbs  track  hang  daboo7  surv  01/2:  ecu  7 

S0ie9  DS-S-0165  0019  00  FRONT/BACK  RADAR  RANGE  MASKDAB007  CM  *01/2:  SO  5C 

S0255  BG  LOSS  Or  DATA  ON  MULTI  D7SS  DAB007  CO/TM  01/2"  ~0G  OS 

NOOOS  DS -N-0CC4 -0036  00  INC  BIT  IN  SuRvEILL  FILE  DAB007  SURV  01/1!  BOU  7 

SO  1 93  DS-S -0146- 0003  00  FPROR  IN  DOWNLINK  ELM  FROC  DAE007  CM  02/0.  BOU  7 

SO  194  DS- S-0145-0002  00  LOSING  ALL-CAIL  SYNC  DABC07  CM  02/0.  S0K10 

SOI  95  DS-S-0144-0001  00  BAD  BIAS  RFG1STER  DAB007  SYS  02/0.  SOU  55 

SO  1 87  DS  S-01 42-00  79  00  UPDATED  CAL  CURVES  DAE007  SA  01/2T  eOMlO 

NOOOS  DS- N- 0001 -0C4S  35  SUFTWR  TESTS  TOR  MBL  7  MBH  DAB007  SURV  Ol/l!  B0M10 

S0202  DS-A-0001  -0001  00  IMPLEMENT  INTERIM  ATARS  DAB008  fOYTRS  *02/0'  80M10 

N0007  DS-S-0 169-0023  00  REQUESTED  SOF TWR  CHNGS  DAB007  MTD  02/0'  BCE  5S 

S0208  DS-S-0150-0005  00  LOCAL  SENSOR  SECONDARY  DAB007  NM  02/K  80U  7 

S0207  DS  S  0149-0003  00  SYSTEM  SPECIAL  MODE  FLAG  DAB007  NM  02/1 :  80U  7 

N2001  8K  RESTARTING  ARIES  DA3006  SURV  02/2!  PCD  OS 

50217  US  LOSS  OF  ATCRBS  TARGETS  DA8007  SURV  02/G!  80W  OS 

S0210  SS  ATCRBS  REM  TRK  DATA  PASSED  DAB007  NM  02/2:  SOS  OS 

50213  DS  S--OI 52-0006  OOINCORR  DISSEM  OE  CIDIN  MSGS  DAB007  COMM  02/2'  80G  5S 

S0212  DS-S -01 51 -0005  OOHI  PR  I  CGMM  BUFF  BACK-UP  DAB007  COMM  02/2'  6GG  5S 

50214  DS-S-01 53-0007  00C1DIN  FAIL  TO  CHK  ZERO  VALUEDAB007  COMM  02/2"  BOG  5S 

SO 190  DS-S-01 54-0008  00  CHANGING  EXT  DATA  STREAMS  DAP007  NM  ##01/22  80  3 

N0008  DS -N-0003-0008  00  CHANNEL  MGT  INTERPOG  SPAC  DAB007  8A  #01/2'  S0M10 

T0001  DS-U-0001 -0084  00  TRACK  ALERT  MESSAGE  DAB006  NM  #02/l:  80S  4S 

S0244  DS-S-01 57-0037  OOINCOR  LINK  SWITCH  CONTR  DAB007  BA  03/0-  BOG  5S 

50218  DS-S-01 56-0010  00  COMPILE  ERROR  IN  DABOOB  RELDAB007  SYS  #02/2'  SOM  55 
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TWARF  TROUBLE  REPORTS  -  DABOO? 

SHEET  6 

I  1R* 

CMC  PROP  «  IN1T 

DEtCRIPT ION 

PROD 

PART  NO  UP?' 

i  bT : 

N0009 

DS-N  0006-001 1 

00 

TEST  28. RUN  4  10CK0UT  PROB 

DAB007 

NM 

03/  1  - 

90S 

bD 

N00 1  0 

JD 

DABS  LOCKOUT  PROB 

DA BOO 7 

NM 

**03/ 1 ■ 

BO 

1 

N001  1 

DS-N-0008- 00 1 2 

00 

NAFEC  REQ  FOR  PRIMARY 

DAB007 

NM 

03/1“ 

80 

4 

NOO  12 

JD 

PCPP  STILL  SET 

DAB007 

NM 

**03/ 1 "  • 

EG 

: 

NOO 1  3 

DS -N-G009- 001 4 

GO 

LXTRA  PROCrSSSPEC  MODE 

DAB007 

NM 

♦03/ 1 “ 

60 

4N 

N00?6 

JD 

SF  UPDATE 

DAB007 

NM 

**03/^ : 

80 

1 

NG027 

DS  N  0009-0003 

00 

COMM  RESPONSE  PROBLEM 

DAB007 

DL 

**03/2 . 

80 

3 

SG251 

D5-5  0162-0013 

00 

UNCONNECTED  SENSOR  FLAD 

DAB007 

NM 

02/1. 

PC  £ 

5S 

60257 

DS  -5  0161  0012 

00 

DILAPLE  A 1  REQUEST 

DAB007 

NM 

03/e  : 

80S 

5S 

SG249 

DS-D  0164  -0016 

00 

INCUR  BIAS  REC  SET  IN  C 1 D I NDAB007 

COMM 

03/1- 

'00 

5S 

E0254 

DS-S-0163  001 5 

00 

SITE  ADAPTATION  UPGRADES 

DAB007 

£A 

03/r 

r  CL 

5S 

N0016 

RS 

TRANSMITTER  OVERLOAD  ON  ELMDAB007 

CM 

03/^ : 

FCu 

OS 

N  1  006 

R  S 

MCU  PARITY  ERROR 

DAB007 

PM 

03/1? 

cC  S 

OS 

NOO  1  5 

RS 

TARGET  REPTS 

DAB007 

SURV 

03/2: 

BCE 

OS 

1.001  7 

DO  S  0166  0020 

00 

SENSOR  STOPS  INTERROGATING 

DAB007 

CM 

03/2  . 

EG 

5C 

NOO  1  9 

DE-N  -0015-0028 

00 

LOSS  OF  DATALINK  MSG-AIRCR 

DAB007 

DL 

*03/21 

80 

4N 

i^G20 

DO  N-001 6-0028 

00 

10SS  OF  DATALINK  MSG 

DAE007 

DL 

•03/2: 

80 

AN 

NG021 

RS 

DISSEMINATING  "A"  CODE 

DAE007 

SURV 

03/2. 

BOA 

cs 

N0022 

RS 

COR  OF  FRUIT  REPLIES 

DAB0G7 

SURV 

*03/2: 

80 

2 

N0023 

PS 

D I  SEEM  OF  ALT  OF  ZERO 

DAB007 

SURV 

03/2: 

SCA 

OS 

NC024 

RS 

LOSS  OF  T.EPTS  TO  ATC  FACIL 

DAE007 

SURV 

03/2: 

BOA 

cs 

N0025 

RS 

BAD  REPTS  BEING  D1SSEM 

DAB007 

SURV 

03/2  : 

EC  A 

cs 

60260 

DS -0-0164  0016 

00 

NM  HANG  PROXIMITY  TEST 

DA3007 

NM 

04/02 

80S 

5S 

A0001 

DS  A -0002-00 1 6 

00 

ATARS  VSL  DESIGN  ERROR 

DAB007 

AJAfrS 

04 /o: 

eo 

5A 

A0002 

DS  A-G003-G017 

00 

FRROfi  IN  ATARS  SIMULATOR 

DAB007 

A?  ARS 

04/02 

eo 

2 

AG003 

DS  A- 0004  -  001 8 

00 

ATARS  EPOCH  CYCLE  CHANGE 

DAE007 

ATARS 

o4/o: 

80 

5A 

SOI  41 

FF 

3  COMP  FAIL  CAUSES  4  TH  FAIL  DAB  006 

SYS 

11/1- 

79F 

OS 

N0028 

DS  N-001 7-0029 

00 

CODE  SWAPFING  LOGIC  WRONG 

DAE007 

SURV 

*04/05 

SOW 

5S 

N0042 

RS 

DOUBLE  DABS  REPORTS  GEN 

CAB007 

SURV 

04/G2 

EGA 

os 

NG043 

RS 

INCREASE  OF  ATCRE5  TRACKS 

DAB007 

SURV 

04/0“ 

ecu 

OS 

N0032 

JD 

S  F.  TIME 

DAB007 

SURV 

•03/3: 

BO 

2 

00268 

FF 

UPDATED  RADAR  RE  I Nr  BIT 

DAB007 

SURV 

04/1  5 

ECS 

os 

S0269 

DS  S-0194-0010 

00 

IMPROVED  SITE  ADAPT  TECH 

DAB008 

SYS 

*04/ lc 

SOF 

bS 

N0036 

JD 

UNEXPECTED  PRIMARY  REQUEST 

DAB007 

NM 

*04/ It 

50 

2 

N0037 

DS  -N-001 3-0026 

00 

INLIST  CLEAR 

DAB007 

NM 

*04/1 ? 

cCS 

55 

NG038 

cs 

SENSOR  DROPPED  I  NT  ERR DC AT 

DAB007 

CM 

04/1* 

eow 

CS 

NG035 

JD 

COMM  PROBLEM 

DAB007 

DL 

04/i : 

SCO 

CS 

S0270 

&w 

SITE  AD  FOR  1  OD  TAF  CCNBOt 

DAB 008 

SA 

*04/1* 

90 

1 

N0034 

DS  N  -0014-0027 

00 

SMF  PROBLEM 

DAB007 

NM 

*04/1 : 

80 

4N 

N0031 

JD 

FAADAB  CELL  CHANGE 

DAB007 

NM 

**03/21 

SO 

1 

N0033 

JD 

CONNECTIVITY  PROBLEM 

DAB007 

NM 

**04/ 1 : 

80 

1 

N0030 

JD 

S  F,  UPDATE 

DAB007 

SURV 

##04/22 

80 

1 

N0029 

JD 

USF  PROBLEM 

DAB007 

NM 

**04/i : 

90 

1 

S0271 

DS - S -01 67-0022 

00 

NEW  CAL  CURVE  -  ELWOQD 

DAB007 

BA 

04/  1  c 

sow 

5S 

S0264 

DS  S-0166-0021 

00 

IPC  M  SITE  ADJ  5ITE  DEF 

DAB007 

SA 

04/0E 

ECU' 

5S 

N0040 

DS-N -00 10 -0009 

00 

SYMBOLS  DISAPPEAR  FROM  STC 

DAB007 

RDAS 

*04/22 

ec-B 

55 

S02  79 

DS  5-01 93-0009 

00 

COLD  START-GMBILD  ON  TAPE 

DABOOB 

SYS 

•04/24 

8  Or 

55 

N0039 

EM 

RADAR  ONLY  DROPOUT  ON  STC 

DAB007 

RDAS 

**04/22 

BO 

1 

50250 

DS-S-01 77-0006 

00 

MODIFY  CM  RTNS 

DAB007 

CM 

04/22 

80 

50 

F-0272 

DS-S-01 76  0005 

00 

ERROR  IN  D1SSEM.  -MODE  4 

DAB007 

SURV 

04/22 

80 

52 
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IUARL  tRIJUDLC  HLIMIUTS  -  DADOO^ 

SHEET  7 

:  u<*t 

CHG  PKOP  *  INIT 

DE SCR 1PT  JON 

PROD 

PART  NO  0»  5 

;  ST  : 

‘.02/4 

DS-S-01 75-0004 

00 

SURV  1RANSMIT  ERR  -  MODE  4 

DAE007 

COMM 

04  . : 

50 

5C 

S0273 

DS-S  0174-0003 

00 

HANG  IN  COUP SC  SC KEEN 

DAB007 

PM 

04/. : 

■  0 

If 

S0275 

DS-S-Ol 72-0001 

00 

HANG  ON  MISSING  BRD 

DA3007 

PM 

04/l: 

>*o 

5C 

S0276 

DSS-01 73-0002 

00 

BAD  BIAS  REG 

DAD007 

PM 

04/2: 

80 

5C 

'.0277 

DS-S-01 70-0024 

00 

REMOTE  DATA  ACTIVE  FLAC 

DAB007 

NM 

•04/e: 

58 

SG278 

DS-S- 0 171 -0002 

00 

NOTIFY  A7ARS  UF  ATC  DENSER 

DAP.  008 

PM 

*04/2: 

E  'j':. 

SZ 

80200 

JS 

LOST  SURV  DUR  ELM  UPLINK 

DAB007 

CM 

*04/ 2 : 

80 

—> 

S0282 

BC 

ELM  209  SCENARIO  PROBLEM 

DAB007 

CM 

05/C  . 

'■>  , 

OS 

A0GC4 

DS- A  -0005-0002 

00 

PROC  SKIPPED- ATCKBS/AICUEEDAE008 

AJ-ARS 

*  0  5  /  C 

8CU 

7 

80 28  7 

05 ■ 5-0100-0007 

00 

PM  t.  MODE  4 

DAB007 

PM 

*05  Cl 

SCU 

7 

S0291 

DS  S-01  87-  0025 

00 

premature  data  req  canc 

DAB007 

NM 

*05/ r. 

fa  OS 

CC 

N0044 

JD 

MESSAGE  EXPIRATION 

DAE007 

DL 

#*05/C: 

eo 

1 

U004S 

JD 

COMM  SCENARIO  PROBLEM 

DAB007 

DL 

05/0  = 

faCG 

C-S 

C0002 

JD 

ZENITH  CONE  PROBLEM 

DAB007 

SURV 

*05/  ; : 

50 

cl. 

A  000  7 

D5  A  0008-0006 

00 

RESPONSIVE  GENERATOR 

DAE008 

A  T-ARS 

♦05/z: 

so 

5N 

A0006 

DS - A  -0007-0005 

00 

DETECTOR  DATA 

DAB008 

AIA4S 

♦os/ i : 

£0 

5N 

A00G5 

DS- A-G006-0004 

00 

CEM  DATA 

DAB008 

ALAflS 

*05/ :: 

BO 

5N 

C  0003 

CS-C-0001  0011 

00 

R  DAS  LEATHER  PL  PORTS 

DAE007 

MTD 

*os/ : * 

ECU 

7 

CG004 

JD 

RADAR  REPORT  DISSEMINATION 

CAB007 

SURV 

*05/  1! 

80 

1 

N0046 

UIS 

ATCRES  FRUIT  REJECTION 

DAE007 

SURV 

05.  :• 

-CCA 

C  = 

N0047 

U'S 

ATCRBS  FRUIT  RE JECT ION 

DAE007 

SURV 

05/  1 ; 

BOA 

OE 

8u296 

ss 

INCGRKECT  DA 5 S  TRACK  INIT 

DAD  007 

SURV 

05/2- 

BCE 

CS 

NG049 

JD 

ALL-CALL  LOCKOUT  PROBLEM 

C4E007 

NM 

**05/  Is 

PO 

1 

NG050 

JD 

USF  PROBLEM 

DAB007 

NM 

*05/ 1 : 

6055 

5S 

N0051 

JD 

SENSuR  STATUS  FFuELEM 

DAB007 

AWL 

4i*05/  1 : 

EO 

1 

NO  052 

JD 

AC  AC  GUI  RE  PROBLEM 

DAB007 

NM 

05/:: 

50£ 

OS 

N0053 

JD 

DATA  R  E  G'JEST 

DAE007 

NM 

*05/ 1 : 

SO 

3 

A0008 

DS  A-OOOV-OC/07 

00 

ALTITUDE  DATA  IN  MESSAGES 

DAB008 

A  TAPS 

*05/ I E 

50 

5  N 

N0054 

JD 

ANT  ENNA  FACE  PROBLEM 

DAB008 

SURV 

*05/ Is 

80 

2 

NG057 

CC 

ATCRBS  RADAR  RANGE  MASK 

BAB007 

CM 

4405/2. 

80 

1 

S0297 

DS-S -  0 1 90  0009 

00 

ILLEGAL  0=  CODE 

DAB007 

CM 

*05/2- 

80U 

7 

SG295 

DS  S -0191-0010 

00 

NOT  HANDLING  illegal  op 

DAE007 

SYS 

*  G  5  /  2  : 

sou 

7 

N0060 

US 

COMMAND  ERROR 

DA9007 

DBrX 

*06/cr 

60 

1 

AC-009 

DS  -A  -0010-0008 

00 

DETECT  AND  RESOLUTION  CHNG 

DAE008 

AT7JJ9 

*05>'2Z 

80 

5N 

N0059 

LM 

DAAT  INITIATION 

DAB007 

NM 

**05/2r 

80 

1 

SC  203 

DS  S  0196- 0001 

00 

ATC  FAILURE  MESSAGES 

DABOOB 

PM 

*05/0: 

CQC 

5S 

50303 

US 

ATCRDS  PROCESSING 

DAB007 

SURV 

06/0* 

£  GU 

OS 

N  l  025 

Em 

INCORRECT  V  R  UE A  MAP 

DAB007 

- - - 

-  *06/i: 

£  OF 

OS 

N0055 

CC 

UUVB  SYCHRONI ZAT ION 

DAB007 

— 

-  *05/2; 

eo^ 

OS 

N0058 

LM 

REEL  ECTOR  FILES 

DAB007 

- - - 

-  *05/25 

60 

ON 

80305 

DU 

SIZE  OVERFLOW  FOR  SSOOAX 

DABOOS 

AT  ArtS 

*06 / C ‘ 

c  ?•  w 

03 

80219 

Bui 

L'AEGOB  CHANGES 

DAS008 

- - 

-  *02/25 

so 

—» 

S0310 

DU 

INCORRECT  BLOCK  DATA  INIT 

DAB008 

PM 

♦06/12 

EOH 

os 

50309 

MB 

ATCRBS  CORR  AT  NM 

DAB007 

SURV 

*06/ i : 

80a 

OS 

N 1  023 

EM 

NO  RADAR  FALSE  TGTS 

DAB007 

SURV 

*06/ i : 

cC  5 

os 

00304 

mf 

DATABASE  KLUDGE 

DAE008 

MTD 

*06/2- 

-  r:y. 

5S 

A0010 

NB 

PROX  MESSAGE 

DABOOS 

ATARS 

*05/2r 

SO 

5N 

50306 

MF 

CHANGED  LCF 

DABOOB 

-  *06/01 

PCM 

5S 

50301 

BC 

ELM  TRANSPONDER  PROBLEMS 

DAB007 

- - 

-  *05/2-' 

80 

ON 

SO  300 

BG 

ELM  PROBLEMS 

DAB007 

CM - 

-  *05/2-- 

C  0 

2 

